Developing a software application for a system with a single processor and a single memory can be straight forward. When viewing the source code for such an application, the source code is often very similar to its original algorithm that describes the computable operations needed to be executed, and thus is generally not too burdensome to follow or analyze given the original algorithm. On the other hand, a software application for operation on multiple processors that uses multiple memory areas may be more complex. Such applications are often referred to as “parallel distributed programs,” and there are generally two approaches to developing such programs.
One approach to developing a parallel distributed program is often referred to as “message passing” (“MP”), which is illustrated in FIG. 1a. With this approach, the system is programmed as multiple tasks or threads, X and Y, that operate or execute on multiple processors, Processors 1 and 2, and handle data residing in multiple memories (not shown). The tasks or threads, X and Y, communicate and cooperate with each other by sending and receiving “messages”. This approach allows for the different tasks and threads to operate in parallel and communicate with each other when necessary, which may result in an efficient and high-performing system. However, the source code for such a system may be burdensome to develop because programming the multiple tasks and threads and having them send and receive messages to each other may dramatically change the code structure and data structure of the original algorithm, and hence may be complicated, tedious, and error-prone. The code structure and data structure of the MP-based programs may lose much of their original characteristics. The abilities to preserve these original characteristics are referred to as algorithmic integrity and data structure integrity. MP programs typically do not preserve algorithmic integrity and data structure integrity.
Another approach is often referred to as “distributed shared memory” (“DSM”), which is illustrated in FIG. 1b. In this approach, a memory space, which may span across multiple memories in different processors, Processors 1 and 2, is dedicated for multiple threads or tasks to access, i.e., it is a globally accessible memory space built on distributed memories. Thus, a thread, X, on one processor, Processor 1, can access data in a memory on another processor, Processor 2, without having to establish another thread. Developing parallel distributed programs using this approach is often easier than using the MP approach, because DSM alleviates the need for major changes in code structure and data structure. However, this approach is generally not as efficient, because it may require a transfer of large amounts of data from the memory on the other processor, Processor 2, to the processor having thread X, Processor 1 and thus may not satisfy the need for high performance parallel computing.
Accordingly, improved systems and methods for parallel distributed programming are desirable.